Gaussian Processes for Machine Learning

Gaussian process (GP) models form a new, emerging complementary method for nonlinear system identification. The GP model is a probabilistic nonparametric black-box model. It differs from most of the other frequently used black-box identification approaches as it does not try to approximate the modeled system by fitting the parameters of the selected basis functions, but rather searches for the relationship among measured data. Gaussian processes models are closely related to approaches such as Support Vector Machines, and specially Relevance Vector Machines [1]. Because GP model is a Bayesian model, the output of Gaussian process model is a normal distribution, expressed in terms of mean and variance. Mean value represents the most likely output and the variance can be viewed as the measure of its confidence. Obtained variance, which depends on amount of available training data, is important information distinguishing the GP models from other non-bayesian methods. Gaussian process can be used for model identification when data are heavily corrupted with noise, and when there are outliers or gaps in the training data. Another useful attribute of the GP model is the possibility to include various kinds of prior knowledge into the model, e.g. local models, static characteristic, etc.